You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the recent transformers versions, e,g. >= 4.56.0 , _align_special_tokens(self) is introduced to check the model special token with the tokenizer configuration during training.
But McaModelConfig does not have these attributes. So mcatrainer failed in the process. In order to match this, this PR patches these attributes.
Thanks for this PR. It certainly addresses the compatibility issue with the _align_special_tokens method in newer versions of the Trainer.
However, I've observed that for the the aligned special tokens are not subsequently saved to the hf_config_json. As a result, the operation is performed, but its outcome is never used.
Therefore, I'd like to suggest two alternative paths forward:
Skip the execution of _align_special_tokens altogether for mcore_adapter (by overriding it in mcore_adapter's Trainer), since it's currently ineffective.
Or, make the function truly effective by modifying the logic to ensure the aligned tokens are correctly persisted in the hf_config_json.
@Kuangdd01 👋hello from ROLL！感谢您对我们的关注和贡献
🤝ROLL期待和社区开发者建立更多的联系，共建交流，一起进步！后续项目也会组织定期的活动、给活跃用户发放礼品等
📧欢迎添加我的微信进一步沟通：tt19960718tt
如果您没有微信，也可以通过邮箱联系：tangtang.tt@alibaba-inc.com
👋 Hello from ROLL! Thank you for your interest and contributions to our project!
🤝 We at ROLL are eager to build stronger connections with our community developers. Let's foster collaboration and grow together! Looking ahead, the ROLL project will also organize regular events and offer gifts to active contributors.
📧 For further discussion:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kuangdd01
changed the title
In the recent
transformersversions, e,g. >= 4.56.0 ,_align_special_tokens(self)is introduced to check the model special token with the tokenizer configuration during training.But
McaModelConfigdoes not have these attributes. Somcatrainerfailed in the process. In order to match this, this PR patches these attributes.